Kolmogorov Complexity: Sources, Theory and Applications

The theory of Kolmogorov complexity is based on the discovery, by Alan Turing in 1936, of the universal Turing machine. After proposing the Turing machine as an explanation of the notion of a computing machine, Turing found that there exists one Turing machine which can simulate any other Turing machine. Complexity, according to Kolmogorov, can be measured by the length of the shortest program for a universal Turing machine that correctly reproduces the observed data. It has been shown that, although there are many universal Turing machines (and therefore many possible ‘shortest’ programs), the corresponding complexities differ by at most an additive constant. The main thrust of the theory of Kolmogorov complexity is its ‘universality’; it strives to construct universal learning methods based on universal coding methods. This approach was originated by Solomonoff and made more appealing to mathematicians by Kolmogorov. Typically these universal methods will be computable only in some weak sense. In applications, therefore, we can only hope to approximate Kolmogorov complexity and related notions (such as randomness deficiency and algorithmic information mentioned below). This special issue contains both material on non-computable aspects of Kolmogorov complexity and material on many fascinating applications based on different ways of approximating Kolmogorov complexity.